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The  Honorable  John  Conyers,  Jr. 

Chairman,  Legislation  and  National 
Security  Subcommittee 
Committee  on  Government  Operations 
House  of  Representatives 

Dear  Mr.  Chairman: 

As  you  requested,  we  are  reviewing  the  $2.7  billion  Short-Range  Unmanned 
Aerial  Vehicle  (UAV)  program  to  determine  whether  it  represents  a  valid 
joint-service  effort  that  will  minimize  duplication  among  UAV  programs  and 
whether  test  results  justify  the  imminent  commitment  to  system 
production.  This  interim  report  focuses  only  on  the  system’s  readiness  for 
production.  Our  in-depth  evaluation  is  expected  to  be  completed  early  next 
year  and  will  focus  on  the  detailed  results  of  system  testing  as  well  as  the 
joint-service  aspects  of  the  program. 


Background 


UAVs  are  pilotless  aircraft  resembling  small  airplanes  or  helicopters  that  are 
remotely  controlled  or  preprogrammed  to  be  controlled  by  on-board 
equipment.  (See  fig.  1 .)  The  Department  of  Defense  (DOD)  initiated 
acquisition  of  the  Short-Range  .UAV  in  fiscal  year  1989  as  a  common-service 
system  for  use  by  the  Army,  Navy,  and  Marine  Corps.  The  system  is  one  of 
a  family  of  UAVs  being  acquired  by  DOD  to  provide  the  capability  to  perform 
reconnaissance,  surveillance,  target  acquisition,  and  intelligence  collection. 

The  UAV  Joint  Project  Office  (JPO),  which  manages  acquisition  of  the 
Short-Range  system,  awarded  contracts  in  September  1989  to  two  firms  to 
provide  candidate  systems  for  competitive  testing  in  fiscal  year  1991. 
However,  technical  problems  with  the  candidate  systems  delayed  the 
program,  and  the  competition  continued  into  fiscal  year  1992.  In  June 
1992,  one  firm  was  unable  to  complete  the  competitive  testing  because  it 
could  not  successfully  certify  military  users  to  safely  operate  its  system. 

The  other  firm  completed  the  test  program  in  July  1992  and  was  selected 
as  the  winning  contractor. 
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The  test  program  included  both  technical  testing  and  other  testing  referred 
to  as  “limited  user  tests.”  The  technical  testing,  during  which  the  systems 
were  operated  by  contractor  personnel,  was  conducted  under  the  direction 
of  the  .IPO.  Test  objectives  included  determining  the  technical  performance 
capabilities  of  the  systems  and  verifying  their  readiness  to  enter  the  limited 
user  testing.  The  objective  of  the  limited  user  testing  was  to  determine  the 
ability  of  the  systems  to  satisfy  user  requirements.  During  limited  user 
testing,  the  remaining  system  was  operated  by  Army  and  Marine  Corps 
personnel,  and  the  tests  were  conducted  by  operational  test  agencies  of  the 
services.  DOU  had  n<»t  completed  its  analysis  of  the  test  results  at  the  end  of 
our  work. 
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Under  the  1989  contract,  the  jpo  can  exercise  a  contract  option  for 
production  of  four  to  eight  systenis.  The  Defense  Acquisition  Board  is 
scheduled  to  n\eet  on  September  28, 1992,  to  decide  whether  to  approve 
exercising  the  option.  Program  officials  stated  they  plan  to  procure  only 
four  systems  under  this  option.  Each  system  is  to  include  eight  air  vehicles 
with  payloads,  a  launch  and  recovery  system,  ground  stations  for 
controlling  the  UAV’s  flight  and  processing  information  from  the  uav,  and 
related  equipment  such  as  trucks  to  transport  system  components.  (See 
fig.  2.) 
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Figure  2:  Short  Range  UAV  System  Description 


Mission  and  Flight  Control 


Modular  mission  payloads  (MMP) 

•  Day/night  imagery  (8) 

•  Daylight  (4) 

•  Air  data  relay  (4) 


8  Air  vehicles  (AV) 


HMMVW  with  Ground  data  terminal  HMMWV  with  ground  data  terminal 

remote  video  terminals 


Launch,  and  Recovery 


HMMWV  with  launch  and  recovery  station 


5-Ton  transport  truck 


5  Ton  transport  truck 


10  Kw  generator 


5-Ton  transport  truck 


5-Ton  transport  truck 


Legend: 

HMMWV  -  High  Mobility  Multi-Wheeled  Vehicle 
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The  JPO  plans  to  procure  a  total  of  48  systems;  31  for  the  Army,  14  for  the 
Marine  Corps,  and  3  for  training.  The  Navy  is  no  longer  participating  in  the 
program.  System  deployment  is  scheduled  to  begin  in  late  1994. 

Results  in  Brief 

Our  evaluation  of  the  Short-Range  UAV  program  indicates  that  the  system 
has  not  demonstrated  its  readiness  for  production.  Contrary  to  its  previous 
plans  to  perform  operational  testing  before  limited  production,  DOD  now 
plans  to  start  production  based  on  limited  testing  that  did  not  adequately 
address  several  critical  system  performance  capabilities.  Moreover,  the 
testing  was  conducted  in  an  environment  not  representative  of  where  the 
system  is  to  be  deployed  and  therefore  could  be  misleading.  In  addition, 
adequate  criteria  was  not  established  to  evaluate  the  system’s 
performance.  Thus,  DOD  will  be  committing  to  the  acquisition  of  a  largely 
unproven  system  if  production  is  started  as  now  scheduled. 

Deferral  of  Operational 
Testing  Increases 
Program  Risk 

During  a  previous  review  of  the  Short-Range  program,'  we  expressed  the 
concern  that  DOD  planned  to  start  low-rate  production  before  subjecting 
the  system  to  operational  testing.  However,  during  that  review,  dod  revised 
its  acquisition  strategy  to  provide  for  completing  one  phase  of  operational 
testing  before  system  production. 

Since  then,  because  of  technical  problems  encountered  with  the  competing 
contractor  systems,  the  JPO  deferred  operational  testing  from  April  1991  to 
March  1994  but  did  not  slip  the  production  schedule  accordingly.  Under 
the  current  contract,  the  JPO  plans  to  exercise  two  system  production 
options,  one  in  September  1992  and  the  other  in  September  1993,  before 
operational  testing. 

JPO  officials  stated  that  they  plan  to  procure  only  7  systems  under  the  two 
options.  However,  the  contract  provides  that  the  JPO  can  procure  up  to  20 
UAV  systems,  or  42  percent  of  the  total  program  quantity,  before  initial 
operational  testing  is  performed.  Although  DOD  policy  permits  low-rate 
production  to  provide  systems  for  operational  testing,  none  of  the  UAV 
systems  to  be  procured  are  actually  needed  to  perform  operational  testing. 
According  to  test  officials,  only  two  systems  are  required,  and  they  are 
already  available.  By  awarding  two  production  contracts  before  operational 
testing,  DOD  will  prematurely  commit  to  production  of  an  unproven  system. 


'Unmanned  Aerial  Vehicles:  Realistic  Testing  Needed  Before  Production  of  Short-Range  Astern 
(GAO/NSlAD-90-234.  Sept,  28. 1990). 
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We  recognize  that  dod  policy  does  not  require  operational  testing  prior  to 
low-rate  production.  However,  our  past  work  has  shown  that  beginning 
production  before  operational  testing  is  performed  can  result  in  adverse 
consequences.  For  example,  we  reported  in  1987-  that  beginning 
production  before  operational  testing  had  resulted  in  the  purchase  of 
electronic  warfare  equipment  that  could  not  be  used  for  its  intended 
purpose.  We  also  reported  that  other  electronic  warfare  systems  were 
placed  in  bonded  storage  rather  than  installed  in  aircraft,  while  other 
systems  were  deployed  to  U.S.  combat  forces  despite  being  judged 
operationally  unsuitable  by  DOD  testing  officials.  Furthermore,  our  work  on 
other  systems,  such  as  the  Airborne  Self-Protection  Jammer,  has  also 
shown  that  once  production  begins,  the  desire  to  prevent  a  production 
break  can  overrule  any  subsequent  test  results.^ 


Limited  User  Testing 
Did  Not  Adequately 
Address  Critical 
Performance  Issues 


Because  of  limited  preproduction  testing,  significant  uncertainty  will  exist 
about  the  Short-Range  UAV’s  performance  if  the  initial  production  option  is 
awarded  as  scheduled.  Experience  with  UAVs  during  Operation  Desert 
Storm  demonstrated  the  importance  of  several  performance  issues  such  as 
the  capability  to  launch  and  recover  UAVs  in  a  combat  environment.  Yet,  the 
Short-Range  system’s  limited  user  testing  did  not  address  or  provided  only 
limited  information  on  these  and  other  important  system  capabilities. 


Launch  and  Recovery  Testing  To  avoid  the  need  for  constructing  nmways  in  a  fluid  combat  environment, 
Was  Unrealistic  ^he  stated  requirements  for  the  Short-Range  UAV  specify  that  the  system 

must  be  capable  of  operating  from  an  unimproved  flat  grass  or  dirt  surface 
measuring  200  meters  by  75  meters.  Moreover,  UAV  operations  during 
Operation  Desert  Storm  showed  that  the  need  for  a  hard  surfaced  runway 
is  a  severe  operational  constraint  requiring  considerable  expenditure  of 
resources. 

Nevertheless,  two  nmways  with  improved  surfaces  were  used  during  the 
Short-Range  system’s  limited  user  testing.  (See  fig.  3.)  While  unpaved, 
both  were  graded  and  packed  by  heavy  road  construction  equipment.  No 
operations  were  conducted  from  unimproved  areas. 


^Electronic  Warfare:  Navy/Air  Force  Still  Developing  Separate,  Costly  Radar  Warning  Receivers 
(GAO/NSlAD-87- 167,  July  1,  1987). 

Electronic  Warfare;  Eiitablished  Criteria  Not  Met  for  Airborne  Self-Protection  Jammer  Production 
(GAO/NSlAD-92-103,  Mar.  23.  1992) 
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The  JPO  Director  acknowledged  that  the  runways  were  improved  surfaces 
but  stated  that  using  engineering  battalions  in  combat  to  construct 
runways  would  meet  requirements.  However,  during  Desert  Storm  the 
Army  found  that  rather  than  using  scarce  engineering  equipment  to 
hurriedly  construct  UAV  airfields  in  a  combat  environment,  it  needed  a  UAV 
system  that  could  operate  from  unimproved  surfaces.  Whether  the 
Short-Range  system  will  fulfill  this  need  is  uncertain;  no  tests  were  done  to 
determine  whether  the  system  could  meet  the  stated  requirement. 

Electromagnetic  Electromagnetic  compatibility  refers  to  the  capability  of  electric  or 

Compatibility  Testing  electronic  systems  to  operate  in  their  intended  environments  without 

Omitted  causing  or  suffering  from  interference  with  other  systems,  and  DOD  policy 

requires  that  systems  be  designed  to  have  this  capability. 
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The  importance  of  electromagnetic  compatibility  was  underscored  during 
Operation  Desert  Storm.  The  JPO  attributed  the  loss  of  two  Pioneer  UAVs 
and  severe  damage  to  two  others  to  electromagnetic  interference. 
Additional  instances  of  electromagnetic  interference  caused  by  emissions 
from  microwave  towers  and  other  systems  also  resulted  in  temporary  loss 
of  control  of  the  air  vehicle  and  prevented  imagery  from  the  vehicles 
payload  from  being  transmitted  to  the  ground  control  station. 

Nevertheless,  the  Short-Range  UAV’s  limited  user  tests  conducted  prior  to 
limited  production  did  not  address  this  issue.  In  fact,  to  avoid  any  potential 
interference  problems,  test  officials  took  steps  to  ensure  that  the  test  area 
was  free  from  electromagnetic  interference. 


UAV  Survivability  Not  DOD  policy  designates  system  survivability  as  a  critical  system 

Evaluated  characteristic  and  provides  that  it  will  be  an  essential  consideration  during 

the  acquisition  of  systems.  The  policy  also  provides  that  developers  and 
test  agencies  will  assess  system  survivability  and  critical  survivability 
characteristics  as  early  as  practical.  Threats  to  be  considered  include 
conventional  weapons,  such  as  anti-aircraft  weapons,  and  advanced 
threats,  such  as  high-power  microwave  and  directed-energy  weapons. 

Nevertheless,  survivability  of  the  Short-Range  UAV  was  not  addressed 
during  limited  user  testing.  In  fact,  DOi)  has  not  established  any 
survivability  requirements  for  the  system  despite  its  stated  policy. 

Although  data  were  collected  pertaining  to  survivability,  such  as  the 
capability  of  threat  weapons  to  engage  the  UAV,  the  data  were  not  evaluated 
and  are  not  to  be  considered  in  making  the  production  decision.  The  JPO 
plans  to  address  the  system’s  survivability  through  an  upgrade  program  to 
be  implemented  in  1994,  after  award  of  the  first  two  production  options. 


System  Reliability  Not  uav  reliability  was  one  of  the  main  concerns  stemming  from  Operation 

Evaluated  Desert  Storm.  Nevertheless,  the  reliability  of  the  Short-Range  uav  was  not 

evaluated  during  limited  user  testing.  Although  limited  reliability  data  were 
gathered  during  limited  user  testing,  no  reliability  requirements  have  been 
established  that  the  system  must  meet  to  enter  low-rate  production. 
Consideration  of  system  reliability  as  a  performance  requirement  will  not 
occur  until  the  full-rate  production  decision. 
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System  Maintenance  Operation  Desert  Storm  revealed  problems  in  maintaining  the  Pioneer 

Performed  by  Contractor  system  even  though  the  maintenance  was  performed  by  contractor 

technicians.  Although  DOD  plans  for  the  Short-Range  UAV  to  be  maintained 
by  military  personnel,  contractor  technicians  maintained  the  system  during 
limited  user  testing.  Testing  of  the  capability  of  military  personnel  to 
maintain  the  system  will  not  occur  until  after  production  begins.  Thus,  DOD 
plans  to  begin  production  without  demonstrating  that  military  personnel 
will  be  able  to  maintain  the  Short-Range  UAV  adequately  to  support 
sustained  operations. 


The  stated  endurance  requirements  for  the  Short-Range  UAV  system  specify 
that  it  must  be  capable  of  providing  surveillance  of  a  target  or  search  area 
for  a  minimum  time  period  on  successive  days.  (The  specific  time  period  is 
classified.)  However,  only  1  of  the  Short-Range  system’s  10  test  flights  was 
of  sufficient  duration  to  measure  against  the  minimum  time  period.  No 
attempt  was  made  to  determine  whether  th>’  system  can  meet  this 
requirement  on  successive  days.  Thus,  the  capability  to  meet  the 
endurance  requirement  is  uncertain. 


System  Endurance 
Requirement  Not  Fully 
Tested 


Preproduction  Testing 
in  Unrealistic 
Environment  Farther 
Compounds 
Performance 
Uncertainties 


Testing  in  a  realistic  environment  representing  combat  conditions  is  dod’s 
primary  means  for  predicting  weapon  system  performance.  However, 
based  on  deployment  plans  for  the  Short-Range  system,  the  limited  user 
testing  was  not  done  in  a  realistic  environment.  Therefore,  the  test  results 
may  not  be  a  valid  indicator  of  the  system’s  capability. 

Realistic  testing  of  the  Short-Range  system  would  require  that  it  be  tested 
at  locations  resembling  those  where  it  might  be  used  in  combat.  According 
to  its  deployment  plan,  the  system  is  required  to  operate  in  areas  marked 
by  hilly  or  mountainous  terrain  with  dense  forests  and  other  vegetation  and 
by  varied  climatic  conditions  such  as  cloudy  weather,  rain,  snow,  and  other 
factors.  Thus,  the  system  is  supposed  to  be  capable  of  performing  its 
mission  under  a  variety  of  environmental  conditions. 


However,  the  Short-Range  system’s  limited  user  testing  was  done  only  in  a 
desert  environment.  The  test  facility,  located  at  Fort  Huachuca,  Arizona,  is 
typical  desert  terrain  for  the  most  part  with  only  sparse  vegetation.  (See 
fig.  4.)  Under  these  test  conditions,  targets  did  not  seem  difficult  to  detect. 
We  viewed  the  test  area  from  a  helicopter,  including  the  targets  that  the 
Short-Range  system  was  supposed  to  identify  during  the  test  program.  We 
had  little  or  no  difficulty  in  readily  identifying  the  targets,  which  included 
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artilleiy  weapons,  surface-to-air  missiles  and  associated  radars,  and  similar 
weapons.  Thus,  the  testing  done  to  date  does  not  address  the  system’s 
ability  to  find  targets  under  a  variety  of  environmental  conditions. 


We  reported  on  this  issue  in  1990  and  recommended  that  the  Secretary  of 
Defense  require  that  operational  testing  of  the  Short-Range  UAV  be 
conducted  in  diverse,  realistic  environments  to  provide  reasonable 
assurance  that  it  would  meet  requirements  before  permitting  limited 
production.  DOI)  disagreed,  stating  that  an  adequate  evaluation  of  the 
system’s  capability  could  be  accomplished  without  testi  .g  in  all 
environments  in  which  the  system  might  be  deployed.  Nevertheless,  DOD 
indicated  that  it  was  in  the  process  of  identifying  an  additional  test  site, 
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with  environmental  conditions  different  from  those  at  Fort  Huachuca,  for 
operational  testing  to  be  done  after  the  production  decision. 

At  that  time,  DOD  also  pointed  out  that  the  tests  were  scheduled  to  occur 
during  Fort  Huachuca’s  rainy  season  during  which  low-hanging  clouds  and 
periods  of  intense  rain  could  be  expected.  However,  Army  test  officials 
informed  us  that  rain  occurred  during  only  one  of  the  Short-Range 
system’s  flight  tests.  The  officials  told  us  there  was  a  brief  period  of  heavy 
rainfall  on  the  Ft.  Huachuca  test  range  on  the  one  day  that  rain  occurred. 
They  said  that  the  operators  maintained  control  of  the  system  during  the 
rain  but  that  the  system  was  unable  to  locate  any  targets.  According  to  data 
provided  by  the  Ft.  Huachuca  Meteorological  Station,  the  total  rainfall  at 
Ft.  Huachuca  measured  one  one-hundredth  of  an  inch  during  June  and  July 
1992  when  the  Short-Range  system’s  flight  test  were  held. 


The  operational  test  agencies  that  conducted  the  test  could  not  fully 
evaluate  most  of  the  test  results.  For  81  of  97  measures  of  performance 
established  for  the  test,  the  operational  test  agencies  only  gathered  the 
related  test  data  because  no  performance  criteria  were  established  for 
evaluation  purposes. 

For  example,  one  of  the  measures  of  perfoimance  was  the  percentage  of 
targets  that  the  Short-Range  UAV  could  recognize  during  the  flight  tests. 

The  test  agencies  gathered  the  results  on  this  measure  of  performance  but 
did  no  evaluation  because  no  criteria  or  threshold  had  been  established  to 
indicate  how  often  the  system  was  supposed  to  be  able  to  recognize 
targets.  This  absence  of  criteria  effectively  prevented  the  operational  test 
agency  from  fully  performing  the  independent  evaluation  of  system 
pert  ormance  that  is  normally  done  and,  in  our  view,  further  compounds  the 
uncertainty  about  the  Short-Range  system’s  performance. 


Adequate  Criteria  Were 
Not  Established 


Recommendation 


We  recommend  that  the  Secretary  of  Defense  require  that  limited 
proc'  iction  be  deferred  until  realistic  operational  testing  provides 
reasonable  assurance  that  the  system  will  perform  satisfactorily. 
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Agency  Comments 


Scope  and 
Methodology 


We  did  not  obtain  official  agency  comments  on  this  report.  However,  we 
discussed  its  contents  with  DOD,  Joint  Project  Office,  and  Army  Project 
Office  officials  and  have  incorporated  their  comments  where  appropriate. 

From  an  overall  standpoint,  JPO  officials  disagreed  with  the  central 
message  of  this  report— that  the  Short-Range  UAV  has  not  been  tested 
sufficiently  to  warrant  a  commitment  to  its  production.  They  maintained 
that  the  system  had  been  thoroughly  tested  and  cited  in  particular  the 
technical  testing  as  having  been  a  comprehensive  evaluation  of  the 
system’s  performance. 

While  the  technical  testing  was  extensive,  we  do  not  agree  that  it  should  be 
the  basis  for  a  production  commitment.  The  technical  testing  was 
essentially  developmental  testing  to  determine  whether  the  system  met 
technical  performance  specifications.  The  testing  was  not  intended  to 
evaluate  the  system’s  potential  operational  effectiveness  or  suitability, 
which  is  the  function  of  operational  test  and  evaluation.  We  believe  that 
DOD  should  have  reasonable  assurance,  based  on  operational  test  and 
evaluation,  that  the  Short-Range  UAV  will  be  operationally  effective  and 
suitable  before  conunitting  to  its  production. 

In  our  evaluation,  we  focused  on  the  limited  user  tests  because  they  more 
closely  approximated  operational  tests.  Even  so,  these  tests  do  not  qualify 
as  operational  tests.  For  example,  the  system  was  maintained  by  contractor 
technicians  during  the  tests  rather  than  by  military  personnel.  Because  the 
Short-Range  system  is  to  be  maintained  by  military  personnel  when 
deployed,  use  of  contractor  technicians  detracted  from  the  realism  of  the 
tests. 


To  accomplish  our  objective,  we  examined  test  schedules,  test  plans, 
performance  requirement  documents,  acquisition  plans,  and  other  records 
bearing  on  the  Short-Range  UAV’s  status  and  potential  effectiveness.  We 
focused  primarily  on  the  plans  for  limited  user  tests  because  they  were 
conducted  by  operational  test  agencies  of  the  military  services  using 
military  personnel  to  operate  the  system.  We  also  examined  DOD  policies 
and  regulations  which  addressed  testing  and  evaluation  of  the  Short-Range 
program  and  discussed  the  system  with  responsible  Department  of 
Defense,  Joint  Project  Office,  and  Army  UAV  Project  Office  officials. 

We  performed  our  work  primarily  at  the  Office  of  the  Secretary  of  Defense; 
Department  of  the  Navy,  Program  Executive  Office  for  Cruise  Missiles  and 
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UAV  Joint  Project;  Army  Missile  Command,  UAV  Program  Office;  Army 
Operational  Test  and  Evaluation  Command;  Electronic  Proving  Ground; 
and  the  Defense  Director  of  Operational  Test  and  Evaluation. 

We  performed  our  review  from  December  1991  to  August  1992  in 
accordance  with  generally  accepted  government  auditing  standards. 

Unless  you  publicly  announce  its  contents  earlier,  we  plan  no  further 
distribution  of  this  report  until  5  days  from  its  issue.  At  that  time,  we  will 
send  copies  to  other  interested  congressional  committees;  the  Secretaries 
of  Defense,  the  Army,  and  the  Navy;  the  Director,  Office  of  Management 
and  Budget;  and  other  interested  parties.  We  will  make  copies  available  to 
others  upon  request. 

Please  contact  me  at  (202)  275-484 1  if  you  or  your  staff  have  any 
questions  concerning  this  report.  M^or  contributors  to  this  report  were 
Jackie  B.  Guin,  Assistant  Director;  Pamlutricia  Greenleaf, 
Evaluator-in-Charge;  and  John  S.  Warren  Jr,,  Evaluator;  Atlanta  Regional 
Office;  and  Charles  A.  Ward,  Evaluator. 

Sincerely  yours. 


Director,  Command,  Control,  Communications, 
and  Intelligence  Issues 


(39S17«) 
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